RSMS engine data flow (`rsms2016`)

This document summarizes how data moves through the worker pipeline (rsms-worker-function), Neutshell, BLTM (bltmdz.exe), and CXPLT (cxplt.exe), and how rsms-legacy (Neutshell C++ sources under rsms-legacy/C++ Projects/neutshell1-…) and the Fortran drivers (rsms-bltm-fortran) fit that picture.

The default run file basename is rsms2016 (run_base in engine_workflow.py / RunConfig). Below, {base} means that stem (e.g. rsms2016).

Note on rsms-legacy: The folder exists on disk when cloned beside this repo (.gitignore may have hidden it previously). Workspace search tools may still omit ignored paths; authoritative Neutshell file creation is mirrored in neutshell-python/neutshell_python/neutshell_dialog.py (ported from the legacy dialogs).

Plain text diagram

[ EXTERNAL DATA ]
  Azure Table scenario row              (spill, duration, timestep, dispersion, river, …)
  RivermileIndex blob                   (stations / mile grid for NFQ merge)
  RMI blob (structure)                   → copied into run folder as {base}.rmi
  Ohio RiverFlows API                    → multi-riverflows-duration (NFQ hydraulic columns)
  EXE_DIR                                → neutshell1.exe, bltmdz.exe, cxplt.exe

[ PYTHON — BEFORE FORTRAN ]
  ctl_builder.write_ctl_and_placeholders
      writes: {base}.ctl
               placeholders: {base}.in, {base}.flw, {base}.dz, empty {base}.nfq  (overwritten next)
  nfq_from_api.generate_nfq_file
      writes: {base}.nfq                        (neutral-flow table consumed by Neutshell)

[ PROCESS — Neutshell ]
  neutshell1.exe  +  {base}.ctl
      reads:  {base}.ctl, {base}.rmi, {base}.nfq
      writes: {base}.in     (BLTM input deck)
               {base}.flw   (BLTM flow / hydro: discharge, area, width, trib Q per grid & step)
               {base}.dz    (dead-zone parameters)
               {base}.fnk   (FINK input for BLTM)
               {base}.fin   (flow info / diagnostics; same stem as CTL)
               {base}.ech   (echo: parsed CTL + run log; stem = CTL with .ech)

[ PROCESS — BLTM ]
  bltmdz.exe  +  <run_folder_path/>
      reads each step: {base}.in, {base}.flw, {base}.dz, {base}.fnk
                         (hydro time series reread from {base}.flw each timestep per driver)
      writes: {base}.out   (BLTM text listing / diagnostics)
               {base}.pcl  (parcel tracking; CXPLT input)
               {base}.mas  (per-timestep mass totals; downstream JSON + optional parity)

[ PROCESS — CXPLT ]
  cxplt.exe  lead trail  {base}.in  {base}.pcl  {base}.   (prefix = path ending with `{base}.`)
      reads:  {base}.in, {base}.pcl
      writes: {base}.TIM   (times / hour axis)
               {base}.PLT   (cross-plot concentrations vs mile/time)

[ PYTHON — AFTER FORTRAN ]
  rsms_results_storage.write_scenario_results_to_azure
      reads:  {base}.TIM, {base}.PLT, {base}.mas
      writes: Azure Tables (PlumeByTime / PlumeByMile) + blobs (edge_timeseries.json, mass_balance.json)
  upload_engine_run_artifacts
      uploads: full run folder (all of the above + logs) under engine-runs/… in blob storage

[ SIDE / LOG ARTIFACTS — worker ]
  neutshell.stdout.log, neutshell.stderr.log
  bltm.stdout.log, bltm.stderr.log
  cxplt.stdout.log, cxplt.stderr.log
  log.txt            (append success lines from each stage)
  log-error.txt      (on failure)

Mermaid diagram

`rsms2016.*` file roles (quick reference)

File	Produced by	Consumed by	Role
`{base}.ctl`	Python (`ctl_builder`)	Neutshell	Control keys: paths to RMI, NFQ, BLTM files, spill geometry, tolerances, …
`{base}.rmi`	Blob copy into run dir	Neutshell	River-mile structure / station layout
`{base}.nfq`	Python (`nfq_from_api`)	Neutshell	Neutral-flow time series merged with index
`{base}.ech`	Neutshell	— (diagnostic)	Echo of CTL lines + run parameters
`{base}.in`	Neutshell	BLTM, CXPLT	BLTM main input; CXPLT reads same deck
`{base}.flw`	Neutshell	BLTM	Hydro fields per grid and timestep (see `bltm.com` `FLOW` definitions in Fortran)
`{base}.dz`	Neutshell	BLTM	Dead-zone file
`{base}.fnk`	Neutshell	BLTM	FINK input
`{base}.fin`	Neutshell	— (diagnostic)	Flow-info companion log
`{base}.out`	BLTM	— (log)	BLTM listing
`{base}.pcl`	BLTM	CXPLT	Parcel output
`{base}.mas`	BLTM	Python parser	Mass-balance columns per timestep
`{base}.TIM`	CXPLT	Python parser	Time axis for plume series
`{base}.PLT`	CXPLT	Python parser	Concentration grid for post-processing

Extensions follow the drivers: BLTM uses lowercase .mas on disk; parsers expect {base}.TIM, {base}.PLT, and {base}.mas (rsms_results_storage/parsers.py). On Windows the filesystem is usually case-insensitive.

Optional / extended CTL keys

The Python Neutshell port supports RMI_INDEX_NAMING_FILE if present in the CTL. The current ctl_builder in the worker does not emit that key; runs rely on RMI_STRUCTURE_FILE and NEUTRAL_FLOW_FILE only.

Primary code references

Orchestration: rsms-worker-function/engine_workflow.py
Subprocess order and required files: rsms-worker-function/exe_runner.py
CTL + placeholders: rsms-worker-function/ctl_builder.py
Result parsing: rsms-worker-function/rsms_results_storage/pipeline.py, parsers.py
BLTM file names in the Fortran driver: rsms-bltm-fortran/.../Bltmdz.for (rsms2016.in, .flw, .dz, .fnk, .out, .pcl, .mas)

RSMS engine data flow (rsms2016)

Plain text diagram​

Mermaid diagram​

rsms2016.* file roles (quick reference)​

Optional / extended CTL keys​

Primary code references​